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Abstract 

In this paper, we address distributed convergence to fair allocations of CPU resources for 
time-sensitive applications. We propose a novel resource management framework where a cen¬ 
tralized objective for fair allocations is decomposed into a pair of performance-driven recursive 
processes for updating: (a) the allocation of computing bandwidth to the applications [resource 
adaptation), executed by the resource manager, and (b) the service level of each application 
[service-level adaptation), executed by each application independently. We provide conditions 
under which the distributed recursive scheme exhibits convergence to solutions of the centralized 
objective (i.e., fair allocations). Contrary to prior work on centralized optimization schemes, 
the proposed framework exhibits adaptivity and robustness to changes both in the number and 
nature of applications, while it assumes minimum information available to both applications and 
the resource manager. We finally validate our framework with simulations using the TrueTime 
toolbox in MATLAB/Simulink. 


1 Introduction 

The current trend in embedded computing demands that the number of applications sharing the 
same execution platform increases. This is due to the increased capacity of the new hardware 
platforms, e.g., through the use of multi-core techniques. An example includes the move from 
federated to integrated system architectures in the automotive industry [^. 

In such scenarios, the need for better mechanisms for controlling the rate of execution of each 
application becomes apparent. To this end, virtualization or resource reservation techniques [I 
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are used. According to these techniques, each reservation is viewed as a virtual processor (or 
platform) executing at a fraction of the speed of the physical processor, i.e., the bandwidth of the 
reservation. An orthogonal dimension along which the performance of an application can be tuned 
is the selection of its service level. It is assumed that an application is able to execute at different 
service levels, where a higher service level implies a higher quality-of-service (QoS). Examples include 
the adjustable video resolutions and the adjustable sampling rates of a controller. 

Typically this problem is solved by using a resouree manager (RM), which is in charge of: (a) 
assigning virtual processors to the applications, (b) monitoring the use of resources, and (c) assigning 
the service level to each application. The goal of the RM is to maximize the overall delivered QoS. 
This is often done through centralized optimization and the use of feedbaek from the applications. 

RM’s that are based on the concept of feedback, monitor the progress of the applications and 


In these early approaches, however, 
proposed an inner loop to control the 


adjust the virtual platforms based on measurements 
quality adjustment was not considered. Instead, reference (8 
resource allocation nested within an outer loop that controls the overall delivered quality. 

Optimization-based resource managers have also received considerable attention |15[[I^ . These 
approaches, however, rely on the solution of a centralized optimization that determines both the 
amount of assigned resources and the service levels of all applications [S 19 20 . In the context of 
networking, reference |13| models the service provided by a set of servers to workloads belonging to 
different classes as a utility maximization problem. However, there is no notion of adjustment of 
the service level of the applications. 

An example of a combined use of optimization and feedback was developed in the ACTORS 
project [^[^. In that project, applications provide a table to the RM describing the required 
amount of CPU resources and the expected QoS achieved at each supported service level [^|^. In 
the multi-core case, applications are partitioned over the cores and the amount of resources is given 
for each individual partition. Then, the RM decides the service level of all applications and how the 
partitions should be mapped to physical cores using a combination of Integer Linear Programming 
(ILP) and first-fit-decrease (FFD) for bin packing. 

On-line centralized optimization schemes have several weaknesses. First, the complexity of the 
solvers used to implement the RM (such as ILP solvers) grows significantly with the number of 
applications. It is impractical to have a RM that optimally assigns resources at the price of a 
large consumption of resources by the RM itself. Second, to enable a meaningful formulation of 
a cost function in such optimization problems, the RM must compare the quality delivered by 
different applications. This comparison is unnatural because the concept of quality is extremely 
application dependent. Finally, a proper assignment of service levels requires application knowledge. 
In particular, applications must inform the RM about the available service levels and the expected 
consumed resources at each service level, increasing significantly communication complexity. 

To this end, distributed optimization schemes have recently attracted considerable attention. 
Reference |22] considered a cooperative game formulation for job allocation to service providers in 


grid computing. Reference |23] proposed a non-cooperative game-theoretic formulation to allocate 
computational resources to a given number of tasks in cloud computing. Tasks have full knowledge 
of the available resources and try to maximize their own utility function. Similarly, in [Tl] the load 
balancing problem is formulated as a non-cooperative game. 

Contrary to the grid computing setup of |22] or the load balancing problem of imi 2^ , this paper 
addresses a lower-level resource allocation problem, that is, the establishment of fair allocations of 
CPU bandwidth among time-sensitive applieations whieh adjust their own service levels. Contrary 
to the cloud computing setup of |23], a game-theoretic formulation may not easily be motivated 
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practically when addressing such lower-level (single node) resource allocation problems. Instead, 
we propose a distributed optimization scheme, according to which a centralized objective for fair 
allocations is decomposed into a pair of performance-driven recursive processes for updating: (a) 
the allocation of computing bandwidth to the applications (resource adaptation), executed by the 
RM, and (b) the service level of each application (service-level adaptation), executed by each appli¬ 
cation independently. We provide conditions under which the distributed recursive scheme exhibits 
convergence to fair allocations). 

The proposed scheme introduces a design technique for allocating computing bandwidth to 
time-sensitive applications, i.e., applications whose performance is subject to strict time deadlines, 
such as multimedia and control applications. In particular, the proposed scheme: (a) exhibits 
linear complexity with the number of applications, (b) drops the assumption that the RM has 
knowledge of application details, and (c) exhibits adaptivity and robustness to the number and 
nature of applications. This paper extends the theoretical contributions of by addressing global 
convergence and asynchronous updates. Furthermore, reference |16] presents the full implementation 
framework in Linux. 

The paper is organized as follows. Section provides the overall framework, while Section 
presents the distributed scheme for resource allocation. Section [^presents the convergence behavior 
for the synchronous and asynchronous case. Section presents technical details required for the 
derivation of the main results in Section]^ Section [^provides selective simulations. Finally, Section 
[^presents concluding remarks. 

Notation: 

• n[a,b] is the projection onto the set [a, h\. 

• For some hnite sequence {xi,X 2 , ■■■,Xn} in M, dehne col{xi, X 2 , ■■■, Xn} to be the column vector 
in M” with entries {xi,X 2 , ...,Xn}- 

• For any x G M, dehne the operator [x]_ as follows: 


X, X < 0 

0, X > 0. 


• For any x G and set A C MA, dehne dist(x,^) = iniy^A H^: — y\\, where || • || denotes the 
Euclidean norm. 

• For some hnite set A, |A| denotes the cardinality of A. 


2 Framework & Problem Formulation 

2.1 Resource manager &; applications 

The overall framework is illustrated in Figure A set X of n (time-sensitive) applications are 
sharing the same CPU platform. Let i be a representative element of this set. Since we allow 
applications to dynamically join or leave, the number n may not be constant over time. 

The resources are managed by a RM that allocates resources through a Constant Bandwidth 
Server (CBS) with period Pi and budget Qi. Hence, application i is assigned a virtual platform 
with bandwidth Vi = Qif Pi corresponding to a fraction of the computing power (or speed) of a single 
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Figure 1: Resource management framework. 


CPU. Obviously, not all virtual platforms Vi are feasible, since their sum cannot exceed the number 
K, of available CPU’s. Formally, we define the set of feasible virtual platforms, (vi,.. ., Vn), as 

n 

V = {v = (ui, ...,Vn) G [0,1]” : < k|. (1) 

i=l 

In this study, the main concern is the computation of the allocation v in real time such that a 
centralized objective is achieved. However, we will not be concerned with the exact mapping of this 
allocation onto the available cores. Such mapping can be performed by a standard first-fit-decrease 
algorithm. Furthermore, in practice, more constraints might be present, especially if applications 
are single-threaded (i.e., they may only run on a single core). In this case, the above feasibility 
constraint will be a relaxed version of the original problem, however, the forthcoming analysis can 
be modified in a straightforward manner to incorporate additional constraints on V. 

Each application i G X may change its service level, Si. It represents a qualitative indicator of 
the delivered quality of application i, assuming sufficient amount of resources Vi. Naturally, it can 
be represented by a real number Si G Si = [Sj,oo) C M, where Sj > 0 is the minimum possible 
service level of application i. The domain Si inherits the partial ordering from M, according to 
which s' < s'- implies that the quality delivered at service level s' is smaller than or equal to the 
corresponding quality delivered at s'f The physical interpretation of the service level may only 
be realized in the context of a specific application. It may represent any quality indicator of the 
application, e.g., the inverse of the accuracy of an iterative optimization routine, the details of an 
MPEG player and the sampling frequency of a controller. We denote s = coljsi,..., s^} the service 
level profile of all applications evolving within S = Si x ... x Sn- 

We implicitly assume here that an application may always increase its service level providing 
the necessary resources, however, in practice it will always be constrained due to the constraints 
imposed in n. Note, finally, that the service level Si is an internal state of application i, i.e., it can 
be written/read only by i. 
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2.2 The matching function 

To be able to assess the performance of a time-sensitive application, it is necessary to introduce a 
performance function. The RM is able to measure at any time t > 0, (a) the soft-deadline of each 
application i, Di(t), which is the time duration of its last CPU reservation, and (b) the corresponding 
job-response time, Ri{t), which is the time elapsed from the start time to the hnishing time of a 
job during its last reservation. A natural dehnition of such performance function for time-sensitive 
applications is the following matching function: 


fi{t) = 


D^{t) 

Ri{t) 


( 2 ) 


Note that fi > —1, a property that will be used often. 

Based on the above dehnition, we dehne a perfect matching between Di and Ri to be the 
situation at which |/j| <5, for some small h > 0. This is the case when application i has the correct 
amount of resources. Instead, a scarce matching describes a situation at which fi < —6, i.e., when 
application i does not have enough resources, and an abundant matching describes a situation at 
which fi > 6, i.e., when application i has more than enough resources. 


2.2.1 Nominal matching function 

The matching function depends indirectly on the virtual platform Vi and the service level Si of 
application i. For a large class of applications, we may derive a nominal representation of the 
matching function, denoted (pi, as a function of Si and Vi as follows: 

(pi{si,Vi) = Pi—- 1, (3) 

for some positive constant Pi. For example, for multimedia applications, the soft deadline Di can be 
considered constant, while the response time can be dehned as Ri = ^i/vi, where Ci = OiSi is the 
execution time per job (at a service level Sj) and Vi is the speed of execution. Similarly, in control 
applications, Ri = where Ci denotes nominal time of execution, while the soft deadline Di is 
considered inverse proportional to the sampling frequency (or service level) Sj, i.e., Di = "i/si. Both 
cases lead to a matching function with the form of (1^. 

It is evident that the nominal matching function satishes the following properties: For some 
Si,s\ G Si and Vi,v[ G V: (PI) Sj / 0 (pi{si,P) < 0, that is, the matching must certainly be 
scarce if no resources are assigned; (P2) Si> s[^ ipi{si,Vi) < ippsPvi), if application i lowers its 
service level, then the performance should not decrease; (P3) Uj > (pi{si,Vi) > (pi{si,vp, if the 
bandwidth of application i decreases, then the performance should not increase. 

2.3 Application weights 

The RM may also assign weights to the applications. We introduce the weight Aj G (0,1] to represent 
the importance that the RM assigns to application i when adjusting its virtual platform Vi. As we 
shall see in a forthcoming section, the weights {A*} will determine the direction of adjustment of 
the virtual platforms {vi} by the RM. The weights {Aj} are considered given and determined by 
the RM. 
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2.4 Fair allocations &; objective 

To define fair allocations, for each application i, we introduce the following nominal fairness 
measure : 


d'i(s,v) = 

-(1 - Vi)Xi [ipi{si, Vi)]_ +Vi'^ Xj [ifjisj, Vj)]_ , (4) 

where vt = is the normalized virtual platform of i over the number of cores. 

The function captures the deficiency in resources of application i compared to the rest of 
applications. When application i is not performing well, i.e., (pi{si,Vi) < 0, and its available 
resources Vi are small, while the rest of applications are performing well, we should expect large 
values for 4>i. 

Definition 2.1 (Fair allocation) For some service level profile s € S, a virtual platform profile 
■V* £ V is fair or balanced if <hj(s, v*) = 0 for all i £ I. 

According to Definition \2.1\ an allocation of virtual platforms v* is fair for application i only 
if V* 0, since at zero resources < 0 and 4j(s, v) < 0. Thus, an allocation v* is fair 

if either (a) r;*)]_ = 0 for all i or (b) [ipi{si,v*)]- < 0 for all i and the ratio of resources 

coincides with the corresponding ratio of weighted matching functions. Since £ [—1, 0], 
case (b) implies that the resources are balanced with the negative performances. For example, if 
is large compared to the rest 1 — vfi then has to be sufficiently negative, i.e., application i 

should not perform so well compared to the rest. Informally, there could not be application i that 
monopolizes the resources at a fair allocation when i performs well and the others do not. 

The above fairness definition introduces a potential centralized problem for fair allocations. 

f mins65,veV /.x 

\ s.t. Fi{si,Vi) = 0, Vi G X. ^ ’ 

However, neither the RM nor application i has complete knowledge of the details of the nominal 
matching function ipi(si,Vi). Thus, on-line centralized optimization is highly prohibited. Instead, 
optimization may only be based upon measurements collected during run-time. 

3 Adjustment Dynamics 

In this section, the centralized objective of fair allocations is decomposed into a pair of performance- 
driven recursive schemes, executed independently by the RM and the applications, thus avoiding 
the computation and communication complexity of centralized optimization. 

3.1 Resource adaptation 

The RM updates the bandwidth Vi = ^i/K, normalized with respect to the number of cores n. The 
unused bandwidth is Vr = k — normalized version is 1;,. = 1 — EEi time 

instances t^, k = 0,1,... the RM assigns resources as follows: 

1. It measures the matching function fi = fi{t}f) for each i £ I, and computes [/j(tfc)]_. 
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(6) 


2. It updates the normalized resource allocation vector v = (ui, .■■,Vn) as follows: 




Vi{tk) + eFi{tk) 


for each i = 1, where Vi = [0, Vk] and Fi{tk) is the observed fairness measure defined as 
follows: 

Fi{tk) = 

-(1 - Vi{tk))Xi[fi{tk)]- +Vi{tk)'^Xj[fj{tk)]-. 

Furthermore, the unused bandwidth is updated according to Vr{tk+i) = 1 — 

3. It computes the original bandwidths by setting Vi{tk+i) = KVi{tk+i). 

4. It updates the time index /c •(— /c + 1 and repeats. 

Note that according to the definition of Fi{tk), if there is a deficiency of resources for i, i.e., 
F'iitk) > 0, then vt will increase, otherwise it will decrease. We consider a constant step size e > 0, 
since it provides an adaptive response to changes in the number of applications. In some cases, we 
will use vector notation, denoting v = coljui, ■■.,Vn} which evolves over V = Vi x ... x Vn- 

Recursion (j^ for the adjustment of resources was motivated by the standard replicator dynamics 
(cf., |24[ Chapter 3]) and in particular the discrete-time equivalent (namely reinforcement learning) 
introduced in j^. Note that the RM time complexity is linear with respect to the number of 
applications, as demonstrated in |16]. 


3.2 Service level adaptation 

The RM provides information to each application i through an observation signal Yi(tk), k = 0,1,..., 
that captures its performance. Applications are designed to adjust their service levels based on Yi(tk) 
as follows: 

Si{tk+i) = [si{tk) + eYi{tk)] , iel. (7) 

A natural selection for the observation signal is to set Yi{tk) = fi{tk)-, he., the observed matching 
function. In this scenario, the application i will increase its service level if fi{tk) > 0, otherwise 
it will decrease it. Alternative observation terms can also be defined with similar properties as 
demonstrated in [^. 


4 Convergence 

In this section, a characterization of the convergence properties of the proposed distributed scheme 
is provided in case of (a) synchronous applications’ updates, and (b) asynchronous applications’ up¬ 
dates. Asynchronous updates constitute a form of perturbation of the nominal synchronous behavior 
which may alter significantly the performance of the scheme. Perturbations due to measurement 
noise are not present, since the RM has direct access to the response time of each application. 
However, internal uncertainties of an application may result in small deviations from its nominal 
matching function. Due to the small probability density of such events, we will not discuss ro¬ 
bustness with respect to such uncertainties, i.e., for the remainder of the paper, we consider 
fi{t) = ip{si{t), Vi{t)), where the nominal matching function satisfies Q. 
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4.1 Feasibility 

The first property of the proposed adjustment process is the feasibility of the resulting virtual 
platforms. 

Proposition 4.1 (Feasible allocations) For sufficiently small step size e = e(n) > 0, the update 
recursion of projected virtual platforms & leads to a sequence of virtual platforms {v(tfc)} which 
satisfies v(tk) G V for all /c = 0,1, ... as long as v(to) £ V. 

Proof. See Appendix □ 


4.2 Minimum guarantees 

The adjustment process guarantees starvation avoidance, i.e., a positive amount of resources (at least 
e > 0) to all applications with non-zero weight. Furthermore, it guarantees a balance condition, 
according to which, in overloaded CPU’s, no application is able to monopolize resources. 

Before stating formally these observations, dehne: 

- L = supi(,x,ken |Fi(4)| < oo, 

- A = minjgi A* > 0. 


Proposition 4.2 (Starvation avoidance) There exists e* = e*(n) < V(l+i)k with e* —)> 0 as 
n —>■ oo, such that for any step size e < e*, inf^gp} Vi{t}f) > e for all i. 

Proof. See Appendix □ 


Proposition 4.3 (Balance) Pick f < ^/k such that maxjgi{/3iKC/sj — 1} < 0. For any 
€ = e(C) < C/L, there exists a number of applications n* = n*{(j) such that, for any set of applications 
X with \X\ > n* and for any i G X, the following hold: 

1. if Vi{to) > (, then Vi{tk) < C after a finite k; 

2. if Vi{to) < (, then Vi{tk) < C for all k = 1,2,.... 

Also, as 0, n*(Q —>■ oo and C^*(C) c, for some positive constant c. 

Proposition |4.3| states that if we pick Q such that Vi < C implies negative matching function for all 

i, and we consider a sufficiently large number of applications n > n*, then all applications will end 
up with a virtual platform less than within finite time. Informally, when the CPU is overloaded, 
no application can monopolize the available resources. Proof. See Appendix]^ □ 


4.3 Synchronous convergence 

In the forthcoming convergence analysis, we will consider either one of the following hypotheses: 
(HI) Let < 1 for all i. 


(H2) Let the nnmber of applications n be sufficiently large such that, there exists 0 < C < 
satisfying properties (1) and (2) of Proposition 4.3 for n*(Q < n. 


Hypothesis (HI) corresponds to the case where the applications are highly demanding, while (H2) 
corresponds to the case where the assigned resources is small compared to the number of applications. 

The asymptotic behavior of recursions can be associated with the limit points of the 

following collection of (nonlinear) ordinary differential equations (ODE): 


( Si{T) 

V 


(pi{Si{T), KVi{T)) 

4>i(s(r),Kv(r)) 


+ Zj(r), i£l, 


( 8 ) 


as the step size e approaches zero, where r refers to the time-index of the ODE. The vector Zj(r) 
represents the vector of minimum length required to drive ni(r) back to Vi and Si{T) back to Si. 
Define (s'^°(-), v'^“(-)) to be the solution of the ODE (|^ starting at (s(ro), v(to)). 

Consider also the linear-time interpolation (sj^e(t), ^^^^(t)) of {(sj(tfc), nj(tfc))}fc, defined as follows: 
Si,e{t) = Si{tk), and Vi^e{t) = Vi{tk), for every tk < t < tfc+i- Introduce also the vector notation 
Se = col{sj,Jj and = coljni^Ji. 


Theorem 4.1 (Synchronous convergence) The following hold: 

1. If either (HI) or (H2) applies, the ODE exhibits stationary points (s*, v*), whieh satisfy: 


s* = Sj, and 


$j(s*, Kv*) = 0, or 
4>j(s*, kV*) > 0,v* = i/k 


VL 


(9) 


2. If either (i) fdiKjs^ — )• 0 for all i, or (ii) n ^ oo, then 

(a) any stationary point of the ODE 0 satisfies: 

s* = Si, n*minj-, |, Vi G X. (10) 

(b) {si{tk),Vi{tk)) (si,Vi) as k ^ oo and e oQ 

Proof. The proof is an immediate implication of a series of propositions presented in detail in 
Section ISTl □ 


In other words. Theorem 4.1 states that stationary points of the ODE ([^ are fair alloeations 
(except for trivial cases where a virtual platform is limited by the size of the core). Furthermore, 
when the CPU is overloaded (i.e., either due to (i) a high demand, or (ii) a large number of 
applications), then the unique fair allocation is a global attractor of the distributed process. 


^By x{t) —>■ A for a set A, we mean limt_>oo dist(x(t),d) = 0. 
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Figure 2: Visualization of asynchronous applications. 


4.4 Asynchronous convergence 

So far, we have implicitly assumed that the adjustment dynamics have synchronized clocks. 

However, the virtual platform, vt, indirectly determines application i’s speed of execution. Hence, 
the update rate of the service level Si varies over time. 

Under asynchronous updates, the asymptotic allocation of virtual platforms may not necessarily 
be fair to all applications. Consider, for example, the case where an application i does not update 
its service level, while all other applications do. Then, under limited available resources, application 
i will retain a sufficiently negative matching function fi, while the matching functions of all other 
applications steadily approach zero. This situation may lead to application i getting asymptotically 
a larger virtual platform independently of its weight Aj. 

To address asynchronous updates, we first introduce the following notation, also visualized in 
Figure 

• t denotes the actual run time; 

• t\. denotes the update instances of application i] 

• tm denotes the update instances of the RM; 

• k{t,i) = {k :t\. <t < denotes the most recent to t update index of application i; 

• fh{t) = {m € N : tm < t < tm+i} denotes the most recent to t update index of the RM; 

• tpiim) = max{m' < m : 3 k s.t. tm' ^ t]. < tm} denotes the most recent update of the RM 
after which the last update of application i occurred. For example, in Figure ifiimit)) = 
fh{t^j^). We set 'ifi{m) = 0 if there exists no k such that t\ < tm- 

• Ni{k) = fh{t\j^f) — rh{tl,) is the number of times that the RM has updated within [t}., 
i.e., between two consecutive updates of application i £ I. 

Throughout this section, we also admit the design assumption that 1 < Ni(k) < N for all i G X 
and k = 0,1, ..., and for some N £ N. In other words, (a) the frequency at which the RM updates 
is larger than the frequency of every application i, and (b) each application i updates with positive 
frequency. Without loss of generality, we will also assume that to = 0, i.e., the RM starts updating 
first. 

Given the above notation, the update recursion of each application i can be written as follows: 

'5*(4+i) = + eli(im(4))]> (11) 

since the computation of the observation signal Yi(-) of each application i is based upon the most 
recent performance measurements taken by the RM at fh{t'^jf). 
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Theorem 4.2 (Asynchronous convergence) Let application i’s actual recursion be given by 
(flip, with 


y.(4) = iv,(A:)-y/(4), 


( 12 ) 


for each i € I, where y/(-) = Then, the conclusions of Theorem 4-i continue to hold. 


Proof. The proof will be shown in Section 5.2 □ 


5 Technical Derivation 


In this section, the technical proofs of Theorems 4.1-4.2 are presented through a series of proposi¬ 
tions. 


5.1 Synchronous convergence (Theorem 4.1) 


The proof of Theorem |4.1| is an immediate implication of the following steps: (a) derivation of 
the ODE approximation of the adjustment dynamics (§-0, provided by Proposition 


5.1 


(b) 


characterization and stability analysis of its stationary points, provided by Propositions |5.2 5.3| 


5.1.1 ODE approximation 

We begin by establishing a connection between the asymptotic properties of the recursions ®-(0 
with the locally asymptotically stable sets of the ODE 0. 

Proposition 5.1 (Synchronous ODE) Consider the recursions @-0. according to which both 
the RM and the applications update synchronously at fixed time instances t^, k = 1,2,.... If A is 
a locally asymptotically stable set in the sense of Lyapuno'^ for the ODE 0), then, for any initial 
condition (s(to), v(^o)) ike domain of attraction of A, (s(tfc), v(tfc)) —)• A as k ^ oo and e —)• 0. 

Proof. The observation signal of the overall recursion is uniformly bounded, and the vector field of 
the ODE 0 is a continuous function on its domain. By [I^ Theorem 1.1] (which shows convergence 
of Euler’s method), we have that for every r > 0: 

lim sup ||(s,(4), W(tfc)) - (s'^o(rfc), Wo(rfc))|| = 0, 
e^0fc=0,...,LT/eJ 

where (s’^°(ro), v'^o(ro)) = (s(to), v(^o)) and Tk = ek. Given that A is locally asymptotically stable 
and the initial condition (s(to), v(^o)) lies within the region of attraction of A, the conclusion follows 
in a straightforward manner by the fact that any solution of the ODE 0 with initial condition 
(s(to), v(^o)) converges to A. □ 


’^See 


14 


Definition 3.1]. 
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5.1.2 Stationary points 


In this section, we characterize the stationary points of the ODE ([^. In general, if an allocation 
(s*,v*) exists snch that (pi{s*,KV*) = 0 for all i, then snch allocation will be a stationary point 
of the ODE ([^ and a fair allocation according to Dehnition 2.1 In sitnations thongh where the 
CPU is overloaded, there might not be snch allocations. The following proposition provides a 
characterization of the stationary points in snch cases. 


Proposition 5.2 (Stationary points) Under either (HI) or (H2), the ODE 0 exhibits station¬ 
ary points satisfying 0, Furthermore, as either (i) Pi/s^i ^ 0 or (ii) n ^ oo, any stationary point 
satisfies (10). 


Proof. If hypothesis (HI) is satished, then (fi{si, KVi) < 0 for all (s, v) and r > 0. In this case, any 
stationary point (s*, v*) satishes ^ which eqnivalently implies that: s* = Sj and 

1 Xi(j)i{s*, KV*) \ 


V* = min 




(13) 


The mapping defined by the second expression of v) is well defined since (j)i{s*,Kv)) < 0 for all i. 
Furthermore, according to Brower’s fixed point theorem Corollary 6.6], it exhibits at least one 
hxed point since it is a continnons mapping dehned on a compact set. The possibility that n* = 0 

Finally, under hypothesis (HI), if we take (di/s^ 


0 for all i, then 


— 1 for all i, which farther implies property 


is exclnded by Proposition 4.2 

vf) _ 

If, instead, hypothesis (H2) is satisfied, then, by Proposition 4.3 there exists a hnite k*, snch 
that, Vi{tk) < C foi' ^11 ■ Hy convergence of Euler’s method, (fi{sD{ t), kvD{t)) < 0 for all 

T > Tfc* = €.k* and all i. Thus, the hxed-point property (13) also applies. Furthermore, if n —?• oo, 
then by Proposition (4.3), C —)• 0, and (/)j(sU(r), KhU(r)) —)• —1 uniformly on r > and z, which 


implies (10). □ 


5.1.3 Global Asymptotic Stability (GAS) 

The following proposition provides a strong convergence property of the ODE (|^. 

Proposition 5.3 (GAS) If either (i) j3i/s^ D or (ii) n ^ oo, then the unique stationary point 
of the ODE satisfying property is globally asymptotically stable in the sense of Lyapunov. 

Proof. See Appendix □ 


5.2 Asynchronous convergence (Theorem 4.2) 

The proof of Theorem |4.2| is a direct implication of establishing equivalence between the synchronous 
and asynchronous update recursions satisfying property (12). In particular, we dehne equivalence 
between two (deterministic) update recursions as follows. 


Definition 5.1 (Equivalent updates) Two update recursions of the form m and observation 
signals {Yi{t\)} and {y({t\)}, i € I, are equivalent if the corresponding linear-time interpolations 
of the updated variables, Sj,e(-) and s^^{-), respectively, satisfy 

limsup |sj e(t) — s' g(t)| = 0. 
t>o ’ 
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In other words, two deterministic update recnrsions of the form © are considered eqnivalent when 
the corresponding linear-time interpolations approach each other nniformly in time as e approaches 
zero. 

We introdnce the following fictitious recnrsion for each i, 


sfitm+l) — [Sj(tm) + (Yi : 


(14) 


for all m > 0. This npdate is synchronized with the time index of the RM and Y-(-) = Note 

that the hctitions observation signals are dehned at times he., at the most recent npdate 

of the RM prior to the last npdate of application i. Since the RM starts npdating hrst, we also set 
Y-(t^.(^m)) = 0 for fol rn snch that = 0, since no performance measnrements are available at 

time t = 0. 

The following proposition shows that, if we pick appropriately the observation signals of the 
(actnal) asynchronons npdate (111, then the asynchronous update becomes equivalent with the 
synchronous update of ([^. 


Proposition 5.4 (Equivalence) For each application i 
sion is given by {11), where YfiFfi) satisfies property (12). 


G X, assume that its actual update recur- 
Then, the following statements hold: 


1 . 

2 . 


The fictitious synchronous update (I 4 ) is equivalent with the asynchronous update 0), 
The fictitious synchronous update (I 4 ) is equivalent with the synchronous update 0 - 


Proof. See Appendix □ 


6 Experimental Evaluation 

6.1 Simulation framework 


To test the assignment of virtual platforms Vi and service levels Sj, the resource management frame¬ 
work was implemented in TrueTime [^. TrueTime is a MATLAB/Simulink-based tool, embedded 
within Simulink, that allows the simulation of tasks executing within real-time kernels. TrueTime 
implements virtual processors through the Constant Bandwidth Server (CBS) [^. Also, it is possi¬ 
ble to adjust the CPU time allocated to the running applications (the bandwidth Vi), as needed by 
our RM. Moreover, TrueTime offers the ability to simulate memory management and protection, 
therefore being a perfect match to simulate our resource management framework. 

A TrueTime kernel simulates a single CPU that hosts the execution of the RM and the CBS 
servers (virtual processors) on top of which the applications are running. The RM observes the 
matching function, fi, of each application i and computes the new reservation Vi according to ([^. 
Observe in this case means that the RM is able to read the start and hnishing time of each job and 
it computes the matching function according to ([^. Then, it updates the virtual platforms and 
communicates to the applications the observations Yi{t\) = fi{t\) according to (12). 

It is here assumed that applications are composed of some time sensitive portions of code, called 
jobs. For example, in a media encoder/decoder a job may be the encoding/decoding of an MPEG 
frame. Applications are requested to inform the RM about the desired duration of each job. Below 
we report a template of the job code. To ease the presentation, we omit some implementation 
details, which can be found in (im. 
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t (sec) 


Figure 3: TrueTime simulation of five applications with different weights that update their service 
levels. 


1 /* j is the job index*/ 

2 id = signal_job_start(j); 

3 adjust = get_performance(j); 

4 /* body of the job. If service aware, it should 

5 modify its resource requirement by adjust */ 

6 do_work(/* parameters */); 

7 signal_job_end(j); 

As the application runs, it is asked to signal the start and the end of a job. This signaling actions 
are performed by invoking respectively signal_job_start and signal_job_end, providing as pa¬ 
rameter the index j of the job. Within the job, the first action is the invocation of the function 
get_perf ormance. This function, which is computed by the monitoring infrastructure, returns a 
measurement of the service level adjustment required to achieve a perfect matching between the 
service level and the virtual platform. Jobs are assumed to be periodic. 

To simulate service-aware applications, we developed a synthetic test application, which performs 
some computation depending on the service level Sj. All jobs of the application have deadline Di and 
are executed in a forever loop. The execution requirement of each job is a linear function aiSi + bi 
of the service level. Hence, applications with a large Ui are more service-sensitive than applications 
with ai close to zero. All applications parameters (ZJj, e, Oj, 6*), which determine the application 
behavior and its capacity to adapt, are set at start time. This enables, for example, the coexistence 
of fully service-aware applications together with service-unaware ones (with a* = 0). 
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6.2 Synchronous updates 

In the first scenario, we consider five applications rnnning in a single core and updating syn- 
chrononsly with the RM according to Q. We pick Ai = 0.9, A 2 = 0.7, A 3 = 0.5, A 4 = 0.3 
and A 5 = 0.1. Each application i has aj= 20, bi= 200 and job deadline Di = 1 msec. The initial 
service levels for the five applications is set to 10 and the applications npdate their service levels 
every time they perform a new job. Finally, we restrict the maximnm assignable bandwidth by the 
RM to 90% to simnlate the fact that the operating system shonld have some space to execnte on 
the same hardware. 

Figure [^reports the quantities measured during the experiment. All applications are gradually 
reducing their service levels as expected due to the negative matching function /j. According to 
Theorem |4.1[ app^ should receive a larger virtual platform compared to the rest of the applications 
due to its larger weight. The final allocation of virtual platforms may not correspond exactly to 


the values assigned by condition ( 10 ), since the conditions partially hold at the beginning of the 


simulation when /j are significantly less than 0. Note though that the relative importance of the 
applications is preserved due to the synchronous updates. 


6.3 Asynchronous updates 

In this scenario, we investigate the effect of the asynchronicity in the final allocation of virtual 
platforms. We consider three different applications starting at the same time, with weights Ai = 0.1, 
A 2 = 0.5 and A 3 = 0.8. Each application uses resources according to aj= 40, bi= 100. Also they 
all have a job deadline Di = 10 msec. Application app^ updates its service level after completing 
10 jobs, while app 2 and app 3 update their service levels after completing 1 job, i.e., asynchronous 
updates are introduced. The initial service levels of the three applications are set equal to 10 and 
the upper and lower bounds on the service levels are set equal to 0 and 20 , respectively. 


Figure 4a reports the quantities measured during the experiment, when applications apply the 


scheme prescribed in the hypotheses of Theorem |4.2| for asynchronous updates. Each application 
employs a constant step-size sequence of e = 0.03. It can be noticed that the RM is able to maintain 
a virtual platform allocation that is consistent with the weights of the applications, while driving 
all matching functions to zero. 

To strengthen the motivation for asynchronicity management, Figure [TE| shows the same simula¬ 
tion when applications do not employ the adjusted observation signal introduced in Equation (pl2), 
and instead they employ the originally introduced observation signal of the synchronous scheme^ ). 
Due to the slow update rate of app^, it maintains a high service level for longer period, which sub¬ 
sequently leads to maintaining a smaller matching function /i for longer period. Thus, even though 
the weight of appj^ is smaller compared to the rest of applications, the RM favors this application 
significantly by gradually providing more resources. This simulation demonstrates that the original 
scheme of Section may not be fair to applications under asynchronous updates, and it was the 


main motivation for the development of the updated dynamics of Theorem 4.2 
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(a) RM deals with asynchronicity. 
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(b) RM does not deal with asynchronicity. 


Figure 4: TrueTime simulation of three applications that asynchronously update their service levels 














7 Conclusions 


We proposed a distributed management framework for allocating CPU resources to time-sensitive 
applications. Given that future computing systems will have to accommodate large number of appli¬ 
cations of different demand levels, resource allocation should not be independent of the applications’ 
performance (a notion captured through the matching function in this paper). Furthermore, given 
that resources are always hnite, applications with higher flexibility in adjusting their demands (or 
service level) should decrease their service levels in overload cases. This paper proposed a distributed 
scheme that incorporates both these two elements, i.e., both measurements of the applications’ per¬ 
formance, and applications’ service-level adjustment. 

In this paper, service-level adjustment is performed by prescribing certain dynamics to the 
applications. Such prescribed response dynamics was the hrst step towards the development of 
a fully distributed allocation scheme. In a fully distributed setup, where applications are not 
prescribed the response dynamics, the question is whether a RM can still be designed that guarantees 
fair allocation of resources independently of the type of applications and their adjustment dynamics. 


A Proof of Proposition 

Let us hrst consider the unconstrained version of the adjustment dynamics (|^, i.e., take k = 1. 
(We will revisit this assumption later.) In this case, the sum of the normalized virtual platforms 
can be expressed as: 


4.1 


^Ui(4+i) - 1 = 
i=l 

n n 

i=l j=l 

Given that —1 < [/j(4)]- < 0, for sufficiently small e = e(n) > 0, the second term of the right- 
hand side is positive for all k = 1,2,.... If ^ 1) then ^ 1- Thus, for a 

bounded number of applications, we may pick sufficiently small e = e(n) such that Vi{tk) G [ 0 , 1 ] 
and < 1 for all A: = 1, 2,.... 

We consider now the constrained version of the recursion (|^. For some time tk, let us assume 
that X]iLihi(4) < 1, he., the allocation is feasible. When we update this allocation using Q, the 
projection operator is activated only if Vi{tk) + eFi{tk) > fo^' some i. Given that hi(4) + 

< 1 (as we showed for the unconstrained dynamics), this quantity may only reduce after 
applying the projection operator. Thus, feasibility is also preserved under the constrained recursion. 

B Proof of Proposition |4.2| 

First, note that '^i[/i(^fc)]- > —n, for all k = 0,1,.... According to (j^, the incremental 

difference of Vi for the unconstrained dynamics, satishes: 

Avi{k) = Vi{tk+i) - Vi{tk) > e {-Xi[fi{tk)]- - nvi{tk)) ■ (15) 

Dehne the set rj(e) = {vi G Vi : 4 G (e, (L + l)e]}, and pick e sufficiently small such that e < 
V(L-i-i)k. Since sup^gpj |Fj(4)| < L, in order for Vi{tk) to approach zero, there should be a time 
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k* at which Vi{tk*) G rj(e). Assuming that Vi{tk) G rj(e), if we pick e sufficiently small, we have 


[fi(tk)\- < l3in{L + l)e/sj — 1. Thus, the right-hand side of (15) further satishes: 

e (-Ai[/i(4)]- - nvi{tk)) > eA - + n){L + l)e^. 

For a given number of applications n, there exists e* = e*{n) < Y(l+i)k with e*(n) —)• 0 as n —>■ oo, 
such that, if e < e*, then the above quantity is strictly positive, i.e., Avi{k) > 0. Since under the 
unconstrained dynamics the lower bound of the projection operator is not reached, the same will 


also hold for the constrained dynamics. From (15), we conclude that if Uj(to) > e for all i, then 
inikenviitk) > e Vh 


c 


Proof of Proposition 


4.3 


At time instance k, let X' C X be the set of applications with resources greater than i.e., I' = 
{i G X : Vi{tk) > C}- Pick 0 < ^ < i/re such that 7 * = — 1} < 0, i.e., all applications 

in T\X' have a negative matching function. Pick also e < C/-^- 

(1) For any i G X', the incremental difference of Vi at k is dehned as Avi{k) = Vi{tk+i) — Vi{tk) = 
^Fi{tk)i assuming that the projection operator in ([^ is not activated. Note that for all j G X\X', 
Vj{tk) < C 

jGi\i iex\x' 

Hence, according to the dehnition of Fi(tk), we have: 


Avi{k) < -e(l - Vi{tk))Xi[fi{tk)]- + e |X\X'| ^*Vi{tk)\ 

< e(l-C) + e(n-L(1-C)/CJ)C7*A, 


where the last inequality results from the fact that —Aj[/i(tfc)]_ < 1, Vi{tk) > C) 1 ~ Vi{tk) < 1 — C 

and |X\X'| > n — [(1 — C)/CJ- Poi' any 


n > ni(C) 


(1-C) -2 + C ' 

C J C7*A 


we have —C < —eL < Avi{k) < —e < 0. In this case, the initial assumption that the projection 
operator in ([^ is not activated is also valid. Furthermore, according to |18[ Theorem 5.1], the 
process Vi{tk) will enter [ 0 ,C] within hnite k. 

(2) For any application i G X\X', the unconstrained incremental difference Avi{k) = eFi[tk) at 
time k satishes: 

Avi{k) < e(l - Vi{tk)) + evi{tkh*X{n - [(1 - C)/CJ - 1), 
since -Xi[fi{tk)]- < 1, 

Ea,[/,(4)]-< E A[/,(4)]_<7*A|X\X'V|, 

j&I\X'\i 

and |X\X'\i| > (n — [(1 — C)/CJ ~ !)■ In order for the process Vi{tk) to exit the set [ 0 ,('], there 
should be a time instance k* at which Vi{tk*) G (C — eX, (^]. For any Vi{tk) G (C — eX, (^], we have: 

Avi{k) < e(l - C + eX) e(C - eL)'y*X{n - [(1 - C)/CJ - !)• 
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If the number of applications satisfy: 


n > n* 2 {Q) = 


1 + 


1-C 

c 


+ 


—2 + (^ — (.L 
(C - eL) 7 *A 


then, Avi{k) < —e < 0 , which implies that the unconstrained process {hj(tfc)} will not exit [ 0 ,C] for 
all future times. The same will hold for the constrained process. 

Finally, by dehning n*(C) = max{n^,n 2 }, both statements (1) and (2) will hold for any n > n*. 
Note that n* —)• oo and ("n* —)> c as (" —)• 0, for some c > 0. 


D Proof of Proposition 5.3 


Let (s*, V*) be a stationary point of the ODE where by property Q satishes st = Sj. Dehne the 
function VF(s, v) = V2(v — V*)T(v_V*) >0. The derivative of W with respect to time r satishes: 


W{s,v) = '^{Vi - 


(16) 


i=l 


Note that at the stationary point and for every i, either one of the following holds: (a) v* = i/k 
and <I>j(s*, Kv*) > 0, or (b) <hj(s*,«:v*) = 0. Note that case (a) implies {vi — v*)'^^i{s, kv) < 0 
for all Vi < i/k and all Sj > Sj. Thus, in case condition (a) is satished for some applications, the 


corresponding additive terms in (16) will be strictly negative. Without loss of generality, it suffices 


to investigate the derivative (16) when all applications satisfy condition (b). Furthermore, it suffices 
to consider the case where Si = s* = Si, since under hypothesis (i) 4>i{si,Vi) < 0 for all Si G Si, 
Vi G Vi and i, and under hypothesis (ii) there exists a time t* after which (l)i(si{T), KVi{T)) < 0, for all 


T > T* (according to Proposition |4.3| and convergence of Euler’s method shown in Proposition 5.1). 


Thus, for all r > r*, we have: 


VF(s*,v) 

n 

= - $i(s*, KV*)) 


i=l 




i=i 


2=1 


E 

2=1 


*|2 


^{vi {viv* - m) ■ 


2=1 




We denote by Ji, I 2 and I 3 the three terms of the r.h.s. of the above expression, i.e., LF(s*,v) = 


h + h + h- Note that: h = -(X)Li ^j) II v - v*|| 2 , I/ 2 I < maxigj{ftK/s* 


V — V 


fr* I I 2 


and 


i/3i < E 


E \"‘"i ~ 


2=1 


r=i 


* “j I 


- Id: - V* 


XjPjK 




< 


i=i j=i 

sup {hi} max I IIIV - v*||i + 

iel I s* J 


I ''j 


i,T>T* 


3 
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f \ 

max < > > 

j&x \ s* ) ^ 


—* 11 — — * 112 

- V II 2 . 


J 


i=i 


Given that ||v — v*||i < \/n||v — v*|| 2 , '^j — ^ A, < 1 for all i, we have 


I/ 3 I < max 1^1 (n sup {uj} + 1 ) 
jGX ^ ^ Y i,r>r* J 


I — — * 112 

|v-v II 2 . 


Under hypothesis (i), i.e., as /?i/sj —)• 0 for all i, and for some fixed size of applications n, the first 
term, Ii, dominates in size the term I 2 +I 3 uniformly in time. Since /i < 0 for any v 7 ^ v*, w e have 
that lU(s*,v) < 0 for any v 7 ^ v*. Under hypothesis (ii), i.e., as n —)• 00 , Proposition 4.3 implies 
that nsupj < Cn(C) approaches a positive constant. Thus, the first term /i dominates 

in size the term I 2 +13 when n —)• 00 . In this case, VU(s*, v) < 0 for any v 7 ^ v* such that Ui < C for 
all i. Thus, under either (i) or (ii), and by [Td} Theorem 3.2], we conclude that the unique stationary 
point (s*, V*), satisfying (|10[), is globally asymptotically stable. 


E Proof of Proposition 5.4 


( 1 ) Let us first consider the unconstrained versions of the actual ( |11[ ) and fictitious update (14). In 
this case, the corresponding linear-time interpolations satisfy for any run time t > 0: 

Si At) - SiAA 

~ ^^^A{t,i)) ~ Si{tfh(t)) 

— l m(t) —1 

/c=0 m=0 

— l rh{t) — l 

fc(t, 2 ) —2 

- ^ eA^i(/c)y/(t^,(^(j|^_^j)), 

k=0 


where the third summation of the r.h.s. summarizes all the observation terms of the RM up to 
time index ^^_^), and the second summation summarizes all the remaining terms, i.e., from 

j)_i) + 1 until m{t) — 1. For example, in Figure the third summation corresponds to all 
time indices up to while the second summation corresponds to the remaining terms up to 

time fh{t) — 1. Since 1 < Ni(k) < N, the latter terms may be zero to maximum N in numbers. 
Also, V'*(hT-(tfc_|_i)) = Thus, we have: 


SiAt) - s'iAt) = 

k{tA) — ‘^ 


/c=0 
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From the last expression, we observe that if = Ni(k)Y-(ff,), then, the first term in the 

r.h.s. becomes identically zero. Given that snpj ^>9 < i, for some £ > 0, we also have 

snPi^i>o 1^(01 < Hence, 

\siAt) - •Si.eWI < eiN + e(m{t) - “ l)^ 

< ei{3N-l), 


which approaches zero as e —)• 0 nniformly in time. Thns, we showed that SiA') and s' ^(•) are eqniv- 
alent according to Definition |5.1| Since the corresponding projected versions are simply trnncations 
to the set Si = [sj, 00 ), the same conclnsion applies for the projected versions. 

(2) Let {SiAkAk denote the service level recnrsion nnder the synchronons npdate Q in order 
to distingnish it from the actnal asynchronons one (11). Let ns also denote sAi') the corresponding 
linear-time interpolation. Similarly to the proof of part (1), it snfhces to consider the nnconstrained 
recnrsions. For every i and t > 0, and since 1 < Ni{k) < N, we have: 


k=ipi{fh{t)) 

< e{m{t) — •ij;i{fh{t))i < eNi, 


which implies eqnivalence of sAi') and sAi')- 
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